| min rank | avg. rank | sentence |
|---|---|---|
| 1754 | 6959.5000 | Parintele spunea raspicat: "Vin vremuri grele". |
| 1207 | 7986.8000 | Premierul italian promite reducerea fiscalit? |
| 1184 | 5057.4286 | Deputatul comunist Igor Dodon continu? campania social? |
| 926 | 8862.5000 | Bogonos Alexandru Irinel - sat Frunzeni, com. |
| 853 | 7141.2000 | Foarte periculoasa Extrema dreapta europeana.. |
| 764 | 12386.2857 | Pai d-aia-i zice minoritate ca-i cantitate neglijabila. |
| 722 | 10239.8571 | Dan-Florian Samarescu Legile reprezinta expresia solidaritatii sociale. |
| 721 | 10383.0000 | Carmen Dragne, medic primar, specialist oftalmolog, Clinica Ama Optimex. |
| 686 | 8879.0000 | Neculai Terente, director Colegiul Economic "Gr. |
| 667 | 11526.5000 | Felicitari curajosilor nostri compatrioti! |
| 448 | 6297.6667 | Contact Peugeot-Citroen prevede scaderea vinzarilor Mar? |
| 423 | 3904.4444 | Etichete aplicare, Finantelor, Fiscul, putut, taxa auto Ministrul Finan? |
| 381 | 12541.5000 | NEEVOLUT adj. 1. primitiv, rudimentar, simplu, (nv.) prost. |
| 351 | 3902.4000 | Avioanele americane folosesc baza militar? |
| 339 | 2752.0000 | Etichete ancheta, membre, Parlamentul, privind, statele membre Mehedin? |
| 325 | 4080.3750 | 09 Iunie 2009, 23:54 Ai perfecta dreptate Aurica! |
| 310 | 1843.0000 | Etichete Craiova, Florentin Tudor, local, sedinta, Tudor Ford Romnia anun? |
| 292 | 11436.7500 | Apropo, sa-i transmiti salutari amicului tau Preafericitul Daniel. |
| 291 | 1435.6667 | Procurorii DNA l-au chemat, ieri diminea? |
| 289 | 5711.1429 | Etichete Buzau, culpa, diagnostic, iulie, medicul Mehedin? |
| 236 | 1929.8000 | Nici Calin Popescu Tariceanu n-a reusit sa-si pastreze vreo functie. |
| 219 | 12350.4286 | Potrivit Legii 370/2004 actualizat?, distrugerea, deteriorarea, murd? |
| 211 | 4078.8182 | Dupa inca vreo 7 etaje, Ion: - Gheo, trebe sa-ti zic ceva.. |
| 194 | 4823.8000 | 13 Februarie 2012, 13:23 Ma bucura decaderea acesti echipa artificiale. |
| 163 | 4748.3333 | Cu alte cuvinte, putem deveni impreuna-creatori. |
| 154 | 7401.8333 | Ne civilizam oameni buni, numai circotiti. |
| 144 | 6163.8750 | Ce facea vicele Eram? Mai merita acestia votati? |
| 144 | 3179.8889 | Rep.: Ce v-a comunicat Alexandru cnd v-a contactat telefonic? |
| 139 | 1678.8571 | Vasile: Gabi, vrei sa-ti spun ceva, Gabi? |
| 132 | 2947.8333 | RO Tag-uri: alba iulia, liga 2, dorinta Steaua Bucuresti ASA Trgu Mure? |
In contrast to subsection 4.5.2.1 we now search for sentences consisting of rare words only. The sentences are ordered by the rank of the most frequent word in a sentence. In the table, we see the corresponding sentences with a minimum length of 40 characters.
The sentences are forced not to contain any everyday word. As a consequence, we get either sentences of some very reduced structure or sentences in some foreign language. Hence, the data are useful for the evaluation of the preprocessing, especially language detection.
select min(w_id)-100 as m, avg(w_id)-100 as a, s.sentence from sentences s, inv_w i where s.s_id=i.s_id and length(sentence)>40 and i.w_id>100 group by s.s_id order by m desc limit 30;
Should we remove the sentences having its least frequent word above some threshold?
4.5.2.1 Maximum word rank in sentence
4.5.2.2 Average word rank in sentence
4.5.2.4 Sentences consisting of many low frequency words II
4.5.2.5 Sentences consisting of short words only I
4.5.2.6 Sentences consisting of short words only II
4.5.2.7 Sentences consisting of long words only I
4.5.2.8 Sentences consisting of long words only II